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THE LAGRANGIAN RADON TRANSFORM AND THE WEIL 
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Abstract. We consider the operator TZ, which sends a function on R^" to its 
integrals over all afHne Lagrangian subspaces in ffi^" . We discuss properties of 
the operator TZ. and of the representation of the affine symplectic group in the 
space of functions on R^". 



pL| , 1- Introduction 



Quantum tomography [121 H] has lately emerged as an alternative picture of 
quantum mechanics [TTl [TH [T7], alternative to the Dirac-Schrodinger picture on 
Hilbert spaces or the C*-algebraic description of Heisenberg and the Wigner-Weyl 
formalism on phase-space. 



^ I The main aim of this paper is to provide a geometrical framework for the tran- 

' sition from quasi-distributions, in the Weyl-Wigner description of quantum me- 

chanics on phase-space, to fair probability distributions [18], in the tomographic 
' description of quantum mechanics. 

'NT ■ 

I From the point of view of the present paper,the essential idea, behind quantum 

tomography, may be illustrated by a simple example. We consider the space of 
04 I lines in the X-rays Radon transform as the space of Lagrangian subspaces when 

the starting vector space is considered to be a two dimensional symplectic abelian 
vector group (phase-space). We may figure out from here that a possible connection 
. between the standard Radon transform and the formulation of quantum mechanics 

I on phase space might exist. Adding the symplectic ingredient to the starting vec- 

. tor space makes available a Fourier symplectic transform and allows to consider the 

family of Lagrangian subspaces and, what is more relevant for the physical inter- 
pretation, a Weyl system [4|. We recall that a Weyl system is a projective unitary 
representation of the abelian vector group with the symplectic structure playing 
the role of the cocycle in the projective unitary representation. 

Roughly speaking, a state, considered as a rank-one projector, is associated, by 
means of the Weyl operators [15], with a function on phase-space, this construc- 
tion is often called " dequantization" . Due to uncertainty relations this function in 
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generic cases will take both positive and negative values and therefore cannot be 
considered to represent a fair probability distribution. However, after integration on 
Lagrangian subspaces, the resulting function will be a fair probability distribution 
(usually called the marginal distribution) as a function on the space of Lagrangian 
subspaces. The aim of quantum tomography is to reconstruct the original state 
when a sufficient family of fair probability distributions is given. In general these 
probability distributions may be replaced with a sufficient set [19) of expectation 
value functions or a quorum of operators. 

These expectation value functions are considered to be directly accessible to an 
experimental determination and this makes more appealing the approach also from 
the point of view of quantum field theory [13' . 




The reconstruction of the states amounts to the inversion of the Radon trans- 
form first to obtain a function on phase space, thereafter we perform a further 
inversion from functions to operators by means of the Weyl map. A relevant point 
is to identify those probability distribution functions on the manifold of Lagrangian 
subspaces which are candidates to represent quantum states once the inverse Radon 
transform is applied to them. 

The present paper will be mostly concerned with the definition of the Radon 
transform on Lagrangian subspaces and its inversion, making it ready for use both in 
classical and quantum tomography,in a coming paper we shall consider the quantum 
framework more closely. It goes beyond previous treatments because the transition 
from lines to generic Lagrangian subspaces turns out to be nontrivial. 



The Lagrangian Radon transform. Consider the space R^" equipped with the 
standard skew-symmetric bilinear form, the group ASp(2n,R) of affine symplectic 
transformations, and the representation of this group in functions on M^". 

The Lagrangian Radon transform TZ is defined in the following way. Let / be 
a function on R^". For any affine Lagrangian subspace L C M^" we consider the 
integral 



This integral transformation is similar to the usual Radon transform (i.e., integra- 
tion over arbitrary fc-dimensional planes in M™), see, e.g. [5], [B], |H], [S]- Various 
operators of this type were considered in literature, the Lagrangian Radon trans- 
form was discussed by Grinberg [7]. However we return to this old problematics. 
Our stand-point is the relation of the Lagrangian Radon transform with the Weil 
representation. 

We suppose n > 1, otherwise some our statements became incorrect. 

Determinantal system of PDE. Decompose M^" as a direct sum R" © M" with 
coordinates (x, y). An affine Lagrangian subspace in general position has the form 




2. Introducing the Lagangian Radon transform 




X = Ty + T, 
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where T is a symmetric n x n matrix and r e M". Consider the following matrix 



ati2 



d 
dtu 



dt 



dt2„ 



d 

St2 



d 



IT- 



(1) 



dU 

\ dri t)T2 

Any function TZf is annihilated by all 3 x 3 minors of this matrix (these equations 
were derived in [7]). 



Let g satisfies this system of PDE. If g has a 'good' behavior at infinity, then 
it has form g = TZf (Theorem 14. 9i see the corresponding statement for the usual 
Radon transform in |5], Theorem 5), however the determinantal system of PDE has 
'extraneous' solutions (see Proposition 16.41 and Section 8). 



We also describe the image of the space of solutions of the determinantal system 
tinder the Fourier transform and show the invariance of local solutions with respect 
to the natural changes of variables (Theorem 14. lip , this is one of numerous exist- 
ing 'higher' analogs of the conformal invariance of the Laplace equation (see [2^ . 
Section 7.8). 



Relation with the Weil representation. It is more-or-less evident, that the 
representation of ASp(2n,R) in L^(R^") is a tensor product of the Weil represen- 
tation of ASp(2n,IR) and the dual representation (see Section 9). This allows to 
construct a continuation of functions TZf to a certain complex domain, see Section 
10. 

An the other hand there are well-known realizations of degenerate highest weight 
representations of semisimple Lie groups in spaces of distributions supported by 
determinantal submanifolds, see Vergne, Rossi |23] (see also an exposition in [22] , 
Section 7.5). We imitate the same construction for the representation of ASp(2n, R) 
in L^(IR^"), see Section 8. We get a certain space of distributions supported by an 
real algebraic variety, that contains two mustaches (as the Whitney umbrella). 



The group ASp(27T.,R). The group ASp(2n,R) is an example of non-semisimple 
groups that are close to semisimple. Such groups (and the group ASp(2n, M) itself) 
were a topic of several works, see [3], [2], [14]. Some standard statements about 
representations of semi-simple groups became non-obvious or even false for such 
groups (for instance, the Radon transform is an intertwining operator, which is 
defined on the Schwartz space and does not admit a continuation to the Hilbert 
space L^(IR^")). On the other hand some new elements appear, see an unusual 
expression for an intertwining operator in Theorem 1 11. 41 

Notation. Let A he a, matrix. 

— A* is the transposed matrix; 

— A is the matrix obtained from A by coefficient-wise complex conjugation; 

— A* = A* is the adjoint matrix 
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— I = I„ is the unit matrix of order n 

— Synim(ri) is the set of symmetric matrices of order n. 

3. The symplectic group 

The complex symplectic group. Consider a linear space W = C^" = C" © C". 
Equip it with the skew-symmetric bilinear form A defined by the matrix 



I 

-I 



(2) 



We denote by Sp(2n, C) the complex symplectic group, i.e., the group of all matrices 

c d I 



preserving the form A. A matrix g ^ ) contained in Sp(2n, C) iff 



a 6\ / l\ (a f I 



c d \-I \c d V-I 



(3) 



We also define the affine symplectic group ASp(2n, C) as the group of affine 
symplectic transformations of C^". It is generated by Sp(2n,C) and translations, 

ASp(2n,C) = Sp(2n,C) k C^". 

The subgroup C^" of translations is a normal subgroup and Sp(2n, C) is the quotient 
group. 

It is convenient to realize ASp(2n, C) as the group of matrices 

(a 6 
c d s 
1, 

of size n + n + 1, where ^ ^ £ Sp(2n,C). Its action on C^" is given by the 
formula 





b 








d 

















The real symplectic group. Two models. For details, see [22], Section 3.2. 
The real symplectic group Sp(2ri, M) is the group of real matrices matrices satisfying 
the same condition ([3|). 

Sometimes it is convenient to realize Sp(2n, R) as the group of matrices of the 
form 

9 -(11)^ (4) 

which are contained in the complex symmetric group Sp(2n, C), i.e., they satisfy 
the condition 
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For matrices of the form Q we can replace the condition ([5]) by the equivalent 
condition 

5 (5 -1)5* = (5 (6) 

Remarks, a) Matrices of the form Q preserve the real linear subspace in C^" 
consisting of vectors > where h ranges in C". 



matrix ( . Also, the pair of equations (O-® implies that g has the block 

structure ^ • U 



b) The condition (jS]) means that our matrices preserve the Hermitian form with 
I 

-1 

$ 

* $^ 

The passage from one model to another is given by the conjugation 

g^KgK-\ ^heie K = ^{j^ ^ ' 'f^ (7) 

Notice that K e Sp(2n, C). 

Consider the affine real symplectic group 

ASp(2n,M) = Sp(2n,R) k R^". 
We realize it as a subgroup in ASp(2n, C) in two ways: 
— as the group of real matrices of the form 

U b r\ , X 

c d s , where " ° e Sp(2n,R) , (8) 
^0 1/ ^ 

or the group of complex matrices of the form 



* $ /i| , where ^ ) eSp(2n,C). (9) 

Subgroups of Sp(2n,M). We consider the following subgroups in Sp(2n,] 

— the subgroup GL(rt, M) consisting of matrices of the form flt-i^ , a is an 
arbitrary invertible matrix; 



I h 



— the subgroup consisting of matrices ^ ) , where 6 is a symmetric 
matrix. This is an Abelian group, ~ r"("+i)/2. 

— the subgroup N - consisting of matrices [ ^ 1 , where c is a symmetric 



c 



matrix. 
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The product N^GL{n,M.)N+ is a dense subset in Sp(2n,M), see, e.g., [22], for- 
mula (1.2.14)). 

Since 

■ I\ A 6\ / I\ _ / I 

the subgroups iV_|_, GL(n, R), and the element ^ ^ together generate the group 
Sp(2n,M). 

We also define a parabolic subgroup P = GL(?i, K.) k N-^-, it consists of symplectic 
a b 



matrices of the form 



a 



t-i 



Next, Sp(2n, M) contains the unitary group U(n). Indeed, consider the Euclidean 
space C" with the standard inner product (•,•). Let us regard C" as a real linear 
space R^". Then Im(-, •) is a skew symmetric bilinear form and unitary operators 
in C" preserve it. 

In the first model the subgroup U(n) C Sp(2n, R) is given by symplectic matrices 
of the form ^ ■ second model, by , where $ is unitary. 

Finally, we define the subgroup 0(n) — GL(7i,R) n U(n). In both models it is 
given by matrices of the form ^ , where a is a real orthogonal matrix. 

Lagrangian Grassmannian. For details, see [22], Section 3.3. Recall that we 
equipped C^" with a non-degenerate skew-symmetric bilinear form A. A subspace 
L is called isotropic if A = on X. A subspace L is called Lagrangian if L is an 
isotropic subspace of maximal possible dimension (i.e., dimL = n). Denote by 
C{n,C) the set of all Lagrangian subspaces in C^". 

Consider the following subspaces in W :— C^" — C" ® C": 

1/+ = C" © 0, = ® C" 

Let T be a symmetric (T = T*) operator V- — >■ V+. Then its graph graph{T) is 
a Lagrangian subspace in V- (B V+ = C'^". Conversely, any Lagrangian subspace 
L C C^" such that L n V+ = has such form. Thus we identify Symm(n, C) with 
an open dense subset in C{n, C). 



The action of Sp(2n, C) on C{n, C) in terms of these coordinates is given by 

T ^ (aT + b){cT + d)-\ (10) 



a b 
c d 

Indeed, consider a subspace L consisting of vectors {Ty,y) G C" ® C". Then gL 
consists of vectors 

b\ (Ty\ _ f{aT + b)y^ 



^c d) \ y ) \{cT + d)y) 
and we pass to the new variable z — {cT + d)y. □ 
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Note that this construction is not canonicah wc fixed the decomposition W = 
V- ®V+. However, we can replace V_ and V+ by an arbitrary pair of transversal 
Lagrangian subspaces Y, Z. The bilinear form A determines a nondegenerate pair- 
ing Y X Z C hy {y, z) 1-^ A{y,z), therefore Z is canonically isomorphic to the 
dual space Y' of Y. Hence for any operator T : Y Z the transposed operator 
T* is also an operator Y Z. Therefore the condition T = makes sense. As 
above, for any symmetric operator T : F — >• Z we get a Lagrangian subspace. 

Our topic is the real Lagrangian Grassmannian £(n,M) C £(n,C). We can 
define coordinates on C{n, M) in two ways. 

First, we can repeat literally the construction given above. Thus we identify 

Symm(n,R) with an open dense subset in £(n, M). Since V- is invariant with 
respect to the parabolic subgroup P, wc have 



However, there is an another way. We can complexify the real symplectic space 
M^" and choose Lagrangian subspaces in the complexification. The most simple 
way is to take W- = graph{iT), W+ = graph{—H). Notice, that complexifying 
a real skew-symmetric bilinear form A we get two forms, the first is the bilinear 
extension of A (we preserve the same notation A): 

A(vi +iv2,w\ +iw2) = A{vi,wi) - A{v2,W2) +iA{vi,W2) +iA{v2,wi) (12) 

The second is the sesquilinear extension 



we divide the form by i to make it Hermitian, M{v,w) = M{w,v). The form M 
is positive definite on W- and negative definite on W+. On the other hand, a 
complexification of a real subspace must be isotropic with respect to both forms, 
A, M. This easily implies that in our case T is unitary. 

Thus we identified £(n, M) with the set of all symmetric unitary matrices. Notice 
that £(n, R) is a U(n)-homogeneous space. 



Two systems of coordinates are related by 

T = [S + i){iS + where T = T* is real, S = 5* is unitary. (14) 

AfRne Lagrangian Grassmannian. Consider the set AC{n,C) of all aflane sub- 
spaces in C^" obtained by shifts of Lagrangian subspace. A subspace of general 
position can be given by equation 

x = Ty + T, where (x, y) e C^", T e Symm(n, C), r e C". (15) 

Lemma 3.1. The action o/ASp(2n, R) on AC{n,C) is given by the formula 



£{n,R) ~ Sp(2n,M)/P. 



M{v,w) = -A{v,w), 



(13) 



£(n,M) ~U(n)/0(n). 




^ {{aT + b){cT + d) 



-1 



ar + r -{aT + b){cT + d)-\cT + s)). (16) 
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Proof. We write 



{aT + b)y + ar + 
(cT + d)y + CT + s 
1 

and pass to the variable z = [cT + d)y + ct + s. □ 





b 








d 




:t)-[ 












The expression in the right hand side of ()16p admits a non-obvious transforma- 
tion: 

Lemma 3.2. // e Sp(2n,C), then 

a-{aT + b){cT + dy^c = (Td* -I- c^)-^ = (cT + d)*-^ 

Proof. 

a - (aT-|-6)(cT-|-d)-ic = 

= {ac-'^{cT + d) - (aT + b)){cT + d)-'^c^ (ac-^d - b){cT + d)-^c. 
Next, by ([3]), we have 

-de' + cd* = 0, ad* - 6c* = 1. 

Therefore 

ac^^d - 6 a(c-id) - 5 = ad*c*-i - 6 = (1 -|- 6c*)c*"i - & = c*~^ 

Finally, 

c*-i(cr + d)-ic^ {Tc' + c-^dc')-^ = (Tc*-l-d*)-i □ 

Let us return to the group ASp(2n,R). For the real realization of ASp(2n, IR) 
formulas are precisely the same. Consider the complex realization of ASp(2n,R) 

Lemma 3.3. Identify >C(n,M) with the set of unitary symmetric matrices as above. 
A subspace v — Sw + a, where w G W-, v G W+, is a complexification of a real 
Lagrangian subspace if and only if 

a = —Sa. 

Proof. A subspace L must be closed with respect to complex conjugation, in 
our coordinates it is given by the formula {v,w) i— > (W,v). Therefore, for any w 
there is h such that 



{Sw + a,w) — {h, Sh + a). 

Therefore, 

S{Sh + a) + cr = h 

and this implies the required statement. □ 

4. Radon transform. Determinantal system 
Denote by 5(R^") the Schwartz space on M^". 
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Radon transform. Consider the space R^" equipped with the standard symplec- 
tic form A. Consider the space AC{n) whose points are pairs 

(Lagrangian afiine subspace L, 

a translation-invariant positive volume form duj on L) 

The space AC{n,M^) is a line bundle over AC{n,R) with fiber M+. 

We define the Radon transform as the map 

nf{L,L,) = J^f{v)dL,{v). 

Obviously, TZf{L,uj) is a smooth function satisfying 

nf{L,t-uj) = t-nf{L,uj), i>0 (17) 

Explicit formulas. Variant 1. Now we wish to specify a form w for a fixed 
L E AC{n,M}. Introduce coordinates of the first kind on the Grassmannian, i.e. 
we identify the set Symm(n,K) x R" with an open dense subset in the AC{n,M.). 
Recall that a subspace L is now the graph of an operator, x = Ty + r, where y 
ranges in R". Thus we have the map R" — > L given by 

y {Ty + T,y). 

We equip L with the pushforward cJAl of the Lebesgue measure on R". 
For / g 5(R^") we define a function on Symm(ri,R) x R" by 

nfiatf{T,T)^ f f{v)dXL{v)^ f fiTy + T,y)dy (18) 

Proposition 4.1. The Radon transform TZ fiat sends the action o/ASp(2ri,R) on 
5(R^") to the action 

(a b r\ 
c d s \ f{T,T) = 
ij 

= f{{aT + b){cT + d)-\aT + r - {aT + b){cT + d)-\cT + s))\det{cT + d)\~\ 

(19) 

Proof. We must evaluate 

UfigL) = ^ f{a{Ty + r) + 6 + r, c(ry + t) + d + s) dy, 

substituting z = (cT + d)y + s, y ^ {cT + d)~'^{z - s) we get (HH). □ 

Explicit formula. Variant 2. Now we equip R^" with the standard inner prod- 
uct. Then any Lagrangian subspace L is equipped with a canonical surface measure 
dfiL induced by the Euclidean structure. 

Remark. In particular, each fiber of the bundle AC AC has a canonical 
representative. Therefore the bundle is trivial. □ 
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Lemma 4.2. Let L be the graph on operator, x ~ Ty + t. Then 

dfj.L = det(l + T^y^/^dXL = I dct(l + iT)\-^dXL. (20) 
If the complex coordinates of L are given by (5, cr), then 

'^"'^ = det\^S+l\ ■ (2^) 

Proof. Without loss of generality we can set r = 0. Consider the element of 
the subgroup U(n) C Sp(2n,M) given by the formula 

_ / (1+T2)-1/2 r(l+2^2)-l/2 

It sends the subspace ® M" to L, 

y^{T{l + T'r'/'y.,{l + TY'^'y) 
and we observe that measures d\L and d/ii differs by the factor det(l + T^)"^/^. 
Next, T and S are related by (O, and (HO]) implies (gl]). □ 
Now we can define the following version of the Radon transform 
'R-compfiS, ^) = y" /(■^) dfJ-L{v), 
where (S, a) are the complex coordinates of L. 

Proposition 4.3. The Radon transform TZcomp sends the action o/ASp(2n,R) on 
^^") to the action 

¥ p f{S,cr) ^ 
1/ 



(22) 



/ (<I>S'+^')(*S'+$)"\ $cr+p -($S'+«')(*S'+$)"'('J'cr+p) • det(*S'+$) 



Proof. In the big brackets we have a change of coordinates. The expression for 
the last factor is a corollary of the following lemma. 

Lemma 4.4. 

^ = |det(¥5 + $)|-i. 

Proof. For a matrix .9=1^ "^1 ^^'^ ^ unitary symmetric matrix S we define 
two expression 

c(g,5):=|det(¥5 + $)|, c{g,S):^^. 

dfJ-gL 

It can be readily checked that c{g, T) satisfies the chain identity 

c(gi52,T) = c(gi, ($2^ + ^2)(^2T + $2)"')c(ff2, T). (23) 
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By definition c{g, S) satisfies the same identity. Obviously, for h G U(n) we have 

c{h,S) = l d{h,S) = l. 
Next, let p,q be contained in the stabilizer P of V- (the corresponding S is 1); 



recall that P consists of matrices ' "^^^^ (1231) 

c{pq, 1) = c{p, l)c{q, 1), c{pq, 1) = c{p, l)c{q, 1). 
All homomorphisms P equals | det(a)|*. It is sufficient to verify the identity 

c{p, 1) = c{p, 1) for p = , this verification is straightforward. 

Next, we represent g as g — hp for h G U(n) and p € P; this is possible since the 
action of U(n) on £ is transitive). By identity (|23|) . we have 

c{hp,l) ^ c{p,l), c{hp,l) ^c{p,l). 

This implies the required statement. □ 

Next, we establish a link between functions TZcompf and TZfiatf- 
Proposition 4.5. 

■ReompiS)=TZfiat{iS + i)itS+l)-\il + iS-Y') ' | dct (1 + i 5) | • 2" . (24) 

/I M 0\ 

Proof. We write the transformation (fT6|) for the matrix i • I I . The 

V 1/ 

factor I det(l + iS')|"i arises from □ 

Partial diflferential equations. Consider the matrix composed of partial deriva- 
tives 

/o d a d d \ 

I ^9tii 0*12 ■ • ■ 9ti„ Oti \ 

h a d 



dti2 ^9*22 '■' dt2n dT2 



a a o a a 

ati„ at2„ ■•■ 9t„„ aT„ 

\ — — — / 

\ ari (9t2 ■ ' ■ aTn / 



(25) 



Each minor of this matrix determines a partial differential operator with constant 
coefficients. 

Theorem 4.6. (5'ee 7 ) For a smooth f G ^(M^") the function (T, r) i-^ 7^/(^, r) 
satisfies the system of equations 

Dnf{T)^0, (26) 
where D ranges in all 3 x 3 minors of the matrix (j25p . 

Proof. We have 

^/ f{Ty + T,y)dy^ [ ( yk-^ + yij—] fiTy + T,y) dy, ii k ^ I 
f{Ty + T,y)dy^ [ (y^^f\{Ty + T,y)dy, 

Otkk JR" JR" \ OXk J 
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Thus the matrix (j25p acts as the following matrix of commuting differential opera- 
tors on the functions / inside the integral: 



dy 



^ dy2 



el: 



dy2 



) 



d d 
dyi dy2 



2Xyj 

dy 



dyn 



9y2 



d 
dyn 















^ dyi^ 




/xi\ 






\ 


Xi 


d 
dyn 

\ J 


^ dyi 




, . . . , Xn 


d 
dyn 

\ ) 


^ dyn 




, 1 


a 

dyn 

\ y 




V 

















= (xi,...,x„,l)'(»(g^,...,5^,0) + (g|^,...,a^,0)'(»(a;i,...,x„,l) 
This has rank 2 and so the determinant of each 3 x 3-minor vanishes. □ 



Schwartz space on AC{n, R). For an element X of the Lie algebra of ASp(2ri, M) 
denote by Lx the corresponding vector field on the space of functions on AC{n, K). 
We say that a C°°-function / on AC{n,M.) is contained in the space if 
for any collection Xi, ... Xk and each M > we have an estimate 

• ..LxJiP)\ = ©(distance between and P)"^^. 

Lemma 4.7. A C°° -smooth function f is contained in S{AC{n,M.)) if and only 
if for any chart of our atlas and for any compact subset C Symm any partial 
derivative of f{T,T) of any order is 0(r)^*^ for all M. 



Proof. Denote the functional space satisfying the condition of the lemma by 

S{AC{n,R))'''\ 

In coordinates, elements Lx are first order differential operators whose coeffi- 
cients are polynomials of degree ^ 2 (formulas can be easily obtained from (|19p ). 
This implies S{AC{n,R)) D S{AC{n,R))^"' . Consider the subgroup in ASp(2n,R) 
consisting of matrices 

'1 b /3^ 

1 
,0 1, 

The Lie algebra action of this subgroup is spanned by the operators and . 
This implies S{AC{n,R)) C 5(yl£(n, M))''*^ □ 

Lemma 4.8. The image of 5(K^") under the Radon transform is contained in 
S{AC{n,R)). 



This is more-or-less obvious, 
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The 7?^/;at-image of the Schwartz space. 

Theorem 4.9. A function f on Symm(n,]R) x M" is contained in the image of 
Ti-fiat if cind only if f is contained in S{AC{n,R)) and satisfies the determinantal 
system ^-1^ of PDE. 

The proof occupy the next section. 

Equations for TZcompf- Now let g — TZcompf be real analytic. Consider its holo- 
morphic extension G to a small neighborhood O of the manifold consisting of all 
{S,a) with 

55*== I, 5 = 5*, CT = -5"V 

in Symm(n,C) x C". 

Theorem 4.10. Let G he such a holomorphic extension. Then the (two-valued) 
analytic function 

det(5)^/^G(5,CT) (27) 
satisfies the determinantal system (|25p - (|26p . 

Invariance of the determinantal system. 

(a b r\ 
c d s \ e ASp(2n,C). Let O C Symm(n,C) x C" be 
l) 

a set such that det (c5 + d) ^ for (T, r) G C If a holomorphic function G{T, r) 
defined onO satisfies the detrminantal system (j25l) - (|26p . then the following function 

G{{aT + b){cT + d)-\ ar + r -{aT + b){cT + dy\cT + s))-det{cT + d) (28) 

also satisfies the determinantal system. 

b) The same statement holds for smooth solutions of the determinantal system 
defined on open subsets on Symm(n,R) x M". 

Theorems 14.101 and 14 . 11 1 are proved in Section 4. 

5. Image of the Schwartz space. Proof of Theorem 14.91 



Fiber-wise Fourier transform on AC{n, M)). Fix inner an product in R^". For a 
subspace Y C K^" denote by its orthocomplement with respect to the Euclidean 
inner product. For a vector ^ we denote by the orthocomplement to the line K^. 

For a function F G S{AC{n,R)) we define its fiber-wise Fourier transform 
"H-compF on S{AC{n,R)) in the following way. Let L be a Lagrangian linear sub- 
space. Let 

^L. 

Then 

Hco^pFi^ + i) - / F{f] + L)e*<«'''> d-n. 
Lemma 5.1. If F e S{A£{n,Rj) then HcompF e S{AC{n,R)). 
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This is straightforward. 

Proposition 5.2. Let F — TZcompf, where f e 5(R^"). For fixed ^ and L orthog- 
onal to ^, the function 'HcompF{^ + L) is independent of L. The function 

viO HcompFiC + L) 

coincides with the Fourier transform f of f 

Proof. Let e R^", and a Lagrangian subspace L is contained in Let 

f{^)= [ /(2;)e*<«'=">dx= /e*"<«'«> / f{s^ + z)dzds = 

e"*<«^«> ( [ f{s£, + u + v)dudvds = 

= [ e''^^'^^ [ nf{s( + v + L)dvds= [ e^<«■''>7^/(77 + i) dr?. 

and we get the desired statement. □ 

Reformulations. Next, we define the same transform in invariant language. 

Denote by AC°{n,M.) the set whose points are pairs {L,£), where L is a La- 
grangian subspace and £ is a linear functional on R^" annihilating L. 

Remark. Notice that a Euclidean inner product on R^" identifies AC{n, R) and 
AC° {n, R) (but this identification depends on a choice of the inner product) . On the 
other hand the skew-symmetric bilinear form A provides a canonical isomorphism 
between L and R^'VL. □ 

For a function F on AC{n,M.) satisfying the homogeneity conditions (|17p we will 
construct a function HF on A£° . 

Fix a linear Lagrangian subspace L C R^" and a volume form w on L. There is 
a well-defined form uj° on R^"/L such that dxdy = uj A lo° . Then 

HfiLJ)^ f e''^^^f{^ + L,Lo)duj°. 

The result does not depend on the choice of w. 

We also write the same transform in the flat coordinates (T, r) e Symm(n, R) x 
R": 

nfiatF{T,v)^ f f{T,T)e'^''' dr. 
It is simply a Fourier transform with respect to a part of coordinates. 

Inverse statement. 

Proposition 5.3. Let F e S{AC{n,M.)) satisfies the determinantal system. Then 
for any ^ 7^ and L orthogonal to ^ the function 

m ■.= HF{i + L) 

is independent of L. 
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Lemma 5.4. Let F satisfies the determinantal system. Then the function g{T, v) = 
'HfiatF{T,v) satisfies the condition 

g{T + S,v) = g{T,v) tfv^O,Sv = (29) 

Equivalently, g depends only on {Tv,v). 

Thus we get a function f(u',v) on M^", w ^ 0, determined by 

KTv.v) = g(T,v). 



Proof. We apply the transform T-Lfiat to the equation 



/n_d d d 



dtkk dt^i d-Tk 



dTk dTi 

and obtain 

We observe that a function (/(T, v) is annihilated by a vector field. The correspond- 
ing flow on Symm(n, M) x R" is given by (T, v)^ {T + eS^\v), where S*"^ is the 
matrix, whose nonzero entries are 



Skk = vf, Ski = sik = ~VkVi, sii = vl 



It satisfies S v = 0. 

Let all entries of v be non-zero. Any symmetric matrix R satisfying Rv = is a 
linear combination of matrices S^'' . Indeed, we have 

^ VkVl J 

The matrix in brackets is diagonal. Therefore it is zero. 

Thus, for V with non-zero entries we have (j29l) . Next, our system (1301) is invariant 
with respect to the transformations T i— )■ ATA^, v Av. Therefore we can send 
each nonzero vector u to a vector with nonzero entries. □ 

Now consider a function F e S{AC{n,M.)) such that /(^ + L) is independent of 
L if L_LC, C 7^ 0. Then we write a function f(^) := + L). 

Proof of the proposition. Indeed, let ^ 7^ 0, L e £(n,M). We chose a 
pair of transversal linear Lagrangian subspaces VL, V+ C K^" in such way that 
Lr\V+ —Q. Then we introduce coordinates {T,t) and apply the lemma. □ 

Smoothness at zero. 

Proposition 5.5. The function f admits a smooth continuation to the point = 0. 



Proof. The condition ([29]) is equivalent to (|30)) . 

We introduce a Z"-gradation in the space of polynomials in variables tki, Vn 
Denote by 5j the natural basis of Z" . We set 

degtki = Sk + Si, degvi = Si. 
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The differential operators ([30|) are homogeneous with respect to this gradation. 
Consider the Taylor expansion of /(T, v) at zero, 

N 

/(r,i;)= ^/(™)(T,«)+i?„(T,«). 

rn—O 

By a homogeneity of the differential operators each polynomial Z'"*^ satisfies the 
equations pop . Therefore the rest R,n{T,v) satisfies the same equations, 

Extend a polynomial Z*^™-' to the complex linear space Symm(n, C) x C". We 
pass to the functions 

It is holomorphic outside the set v — 0. The codimension of this set is n > 1. 
Therefore by the Hartogs theorem it is holomorphic on the whole C". 

Next, 

Consider the function 

Notice that if |T| ^ e, \v\ ^ e, then \Tv\ ^ e^, on the hand any vector w such that 
< £^ can be represented in such form. Therefore 

m{w,v) = oiY,H\ + J2\v,\r/', v^o. 

Hence f is C°° at zero. 

End of proof of Theorem Let F e 5(yl/:(n,R)), f e ^(M^") be the same 
as above. 

Lemma 5.6. Let 

9{^) = / f(C)e'*«^* dx 

Then 

F{t] + L) = TZconipgiv + L), ri±L 

Proof. We can evaluate the Fourier transform /(^) of a function g E iS(M^") 
in the following way. For ^ e K^" consider a Lagrangian subspace L orthogonal to 
^. Then 

fiO - / 7^eo™p(x + L)e^«^'da;. 
6. Invariance of the determinantal SYSTEM. Extraneous solutions 



Proof of Theorem 14.111 Holomorphic case. Let the matrix T = {t^i} range 
in Symm(n,C). Consider the following matrix 
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We say that a function F on Symni(?i, C) satisfies the detcrminantal system if 
it F is annihilated by all (k + 1) x {k + l)-minors of this matrix. 

Theorem 6.1. Let ^ ^ E Sp(2n, C). Let O C Symm(n, C) be an open subset, 

and det(cT' + d) ^ on O. Then for any holomorphic function F on O satisfying 
the system Ak, the function 

F{{aT + b){cT + d)-^) det(cT + d)-^'^ (32) 

satisfies the system A^. 



The statement was formulated in [22], Subsection 8.8.3 without formal proof, 
but modulo consideration of [52], Section 8.8, a proof is straightforward. 

In our case, k — 2. 

Now consider the group Sp(27i, C) C Sp(27i + 2, C) acting (locally) on Symm(rt + 
1,C). We represent an element of Symm(n + 1,C) as (n + 1) x {n + l)-matrix 
Sn Si2 



S2I S22y 

We get an action of Sp(2n, C) given by 



fa b 0\ 

10 

c d 

yo ly 



F 



Sii Si2 

S21 S22 



_ fiaSn+b)icSn+dr^ (a - {aSn + b)icSn + d)-^c)S,2\ ... ^ 

- P [ S2i{cSn + S22 - S2i{cSn + d)-^cS^2 ) •det(«5n + 6) 

Elements of Sp(2n, C) are represented by matrices of size {n + 1 + n + 1). Recall 
that 5*22 is a number, 5i2 is a column, and = Sl2. 

We observe that 



— transformations p send functions independent of 5*22 to functions independent 

of 5'22- 

— the action of Sp(2n, C) on functions depending on ^n, 5*12 coincides with the 
action (p8)) (we set T = Su, r = S'12; since we work with Sp(2n, C), we have r = 0, 
s = 0). 

— the restriction of the system A2 to functions independent of 6*22 is precisely 
the determinantal system (P5|) - (P5)) . 

Thus we proved the invariance of the determinantal system with respect to 
Sp(2?i, C). The invariance with respect to translations is obvious. 



Proof of Theorem 14. Ill Smooth case. Let F be a function defined on an open 
subset in Symm(n,IR) x M". Consider the Taylor decomposition of F at a point 
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f{t,t) - ^p,-,(t,t) + i?,„,,(r,T), 

3=0 



where Pj,s{T, r) is a polynomial of degree kj homogeneous with respect to tki — sm, 
Tk^ o'k, and Rm,s{T, r) has zero partial derivatives of order ^ m at the point {S, a) 

Lemma 6.2. A function F satisfies the determinantal system if and only if for all 
s for all j polynomials pj^s{T,T) satisfy the same conditions. 

Proof. Straightforward. 

Let us prove the theorem. Obviously the determinantal system is invariant with 
respect to translations (T, t) i— > {T + H,t + h) and transformations 

(T, r) {aTa\ ar), a G GL(n, M) 

In other words, the determinantal system is invariant with respect to the parabolic 
subgroup P C Sp(2n. R) consisting of matrices 

b r\ 

a*-i s e ASp(2n.M) (33) 
,0 ij 

Let us add the subgroup N_ C Sp(2n,R) consisting of matrices 

'1 0^ 

c 1 I e ASp(2n,K), (34) 
,0 1, 

it acts by transformations 

F{T, r) ^^ F(T(1 + r)"\ (cT + d)*"V) det(l + T)"\ (35) 

Lemma 6.3. If F satisfies the determinantal system, then for any minor S of (|25p 
the right-hand side of (j35[) is flat at 0. 

Proof. The Taylor polynomial of order m at of the transformed function 
depends only on the Taylor polynomial of order m of F. On the other hand S 
applied to the Taylor polynomial is zero. □ 

Now take a point (5*0, o-q) and element g of ASp(2ri,,R). First, we consider a 
translation r sending (So,ao) to (0,0). Next, decompose gr~^ as a product p ■ h, 
where p £ P and h G N. Clearly, applying a minor S to the transformed function, 
we get a function flat at 9(5*0, ctq)- D 

Modifications of formulas for transformations. Notice that the determinantal 
system is invariant with respect to transformations ([25]) . But expressions p^ . (P^. 
(1^^ have slightly another form, they contain absolute values of determinants. 

1) The case of ([T^ is quite transparent, the expression 

|det(cr + d)| 
det (cT + d) 

is locally constant. 
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2) Let us examine transformations ([22]) . We write 
|det(¥S' + ¥)|~^ = 

= det($¥)-i/2 • det [(I + ^-i^-S*] "^/^ • det [(I + $^^1^5'] (36) 



First, we note that 

det($$)-i/2 ^ det($$*)"i/2 

and we can take the positive value of the square root ($ is non-degenerate). Next 
we evaluate det(-)~^/^ as 

det(I + H)-^/^ := det(l --H+ -H^ + . . .) 

2 8 

and ||i7|| in our case is ^ 1 (see, e.g., [121, Theorem 3.3.5). Thus, for fixed 

^' <f> P \ , the expression p6p is well-defined and holomorphic in a neighbor- 
ij 

hood of U(n) H Symm(n, C) in Symm(n, C). 
We also transform the same expression as 

|det(¥S' + ¥)P^ =±det(¥S'-|-$)-i/2det(*5"^ + $)"^/2 = 

-1/2 

detS J ' 

Therefore, we can write the transformations (1221) as 



,- 1 /det (($S' + ^')(*S' + $)-!! 
±det(*5' + $)-i • ' ' 





(1 










¥ 






[0 








/(5, a) (det 5)1/2 



X det(($S' + «')C¥S' + ¥)"i 
Thus, the twisted function 

f{S,a){detS)'/^ 
transforms according the formula ([28]). 



/(^($5 + 1')(*S' + $)"\$cr + p - ($5 + *)(«' S' + $)"i('J'cr+p))x 

,-1/2' 



det(^'S' + $)" 



3) It remains to examine the transform (j24p from flat to the compact model. 
Again, we transform the last factor, 

I det(l+iS')ri = dct{l+iS)-^/'^ dct(l-iS-^)-^/^ = ± det{iS)^^^ det{l+iS)-^ 

Therefore formula ([M]) comes to the form 
n,ornpiS)=nfiat{{S+t){iS+l)-\il+iS-')-')-{detSy/^det{l+iS)-'-2'' (37) 

Proof of Theorem 14.101 It follows from formula ([57)) . 
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Extraneous solutions. All solutions of the determinantal system in S{AC{n,K.)) 
are contained in the image of the Radon transform. We wish to show that there 
are additional local solutions. 

For instance, any function on£ = U(n)/0(n) determines a function on AC{n,M.), 
which is constant on fibers. We will present additional solutions of the determinan- 
tal system on the two-sheet covering U(n)/0(n)~ of U(n)/0(n) 

Recall that irreducible representations of U(n) are enumerated by integer signa- 
tures 

mi ^ . . . ^ m„. 

The space of functions on C{n, M) is a multiplicity-free direct sum of representations 
of U(n) with signatures (2pi, . . . , 2p„) (see, e.g. [9], [IQ]). Functions 



til ■ ■ -ti 



J|det 



iij 



\tlj ...Zjj^ 

are highest weight vectors of these representations. 

The space of functions on U(n)/0(n)~ also contains representations with signa- 
tures {2pi + l,...,2pn + 1). 

We look for analytic solutions of the (twisted) determinantal system on coverin 
space U(n)/0(ri)~. Recall that we apply 3x3 minors of ((3T|) to a function 

det(T)i/2/(r), 

where / is analytic in a neighborhood of the surface TT* = 1. 

Proposition 6.4. All representations of U(n) with the following signatures are 
contained in the space of solutions 

(2a + 1,2/3 -I- 1,1,...,!), a^p^O (38) 

(2a, 0,..., 0,-27), a ^0,7^0 (39) 

(-l,...,-l,-27-l,-2<5-l), S^j^O (40) 

Notice that solutions of the type (|39p are global solutions on £{n,R), and solu- 
tions ([55]) . pOj) are determined on two-sheet covering. 



Proof. Functions 

^tl2 t22^ 

obviously are annihilated by 3 x 3 minors of the matrix ((25)) . After shift by (1, . . . , 1) 
(see ([27| give us signatures of the form (p8| . We apply the transformation 





7r2 

(it has the form ([32]) with k — 2) and get 
det 



j).9W=5(T-i)det(r)-i 




tl{n-2) ■ ■ ■t{n-2){n-2)/ ■ ■ ■ t{n-l){n-l) 
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This gives signatures of the type (|40| . 
Next, the functions 

til (41) 
are annihilated by 2 x 2 minors of the matrix pip . We apply the transformations 



Ti/(T) 
and get the functions 



I 

-I 



7(T) = 5(T-i)det(r) 



-1/2 



/ ill 



det 



■ tl{n-2) 



detT-"~i/2, 



(42) 



n — 2)(n— 2) y 

they also are annihilated by 2 x 2 minors of the matrix pip . We multiply functions 
of the types and (|1T|) and get signatures of the type ([HS)) . 



7. Inversion formulas 



We wish to reconstruct a function f{x, y) of 2n variables from a function F = TZf 
of n(n + l)/2 + n variables. There are many ways to do this, we propose 3 variants 
(which are straightforward imitations of inversion formulas for the usual Radon 
transform) . 



Inversion 1. As we have seen in Proposition I5.2[ the Fourier transform of / can 
be reconstructed from TZf. 



Inversion 2. Introduce parameters (pi, (pn G [0,27r), and pi, . . . , pn ^ 0. 
For each ip, p consider the afhne Lagrangian subspace L ~ L[(p,p] defined by the 
equations 

I Xj cos ipj + yj sin ipj — pj . 
The set A(n) of all such subspaces is a submanifold AC{n,M.). 
Consider the decomposition 

M2" = M2 ^ . . . X 

where each factor has coordinates (xj,yj). In j-th factor consider a line i[ipj,pj] 
defined by the equation xj cos ipj + yj sin Lpj = pj. The Lagrangian subspace L[ip,p] 
is the direct product of the lines £[tpj,pj], moreover 

d>^L[^,p] = dXi[^^^p^] X • • • X dXi[^^^p.y, (43) 
d^J■L[^,p] = dfii[^^,p^] X • • • X d^i[^^ p.], (44) 

and 

A{n) =A£{1) X ••• X AC{1) 
Restrict TZcompf to A{n). We get the integral transformation 




22 



GIUSEPPE MARMO, PETER W. MICHOR, YURY A. NERETIN 



the left hand side is a function on the product of the n-dimensional torus T" and 
M". This transform is a tensor product of two-dimensional Radon transforms. 
Therefore we can automatically write the inverse operator as a tensor product of 
inverse operators for (see, e.g. [6], [8]. Denote by F the average oilZf over the 
torus 

F{pi. . . . ,pn) = „ / ... / nf{ipi,...,ipn,Pl,-.-,Pn)]dipi...dipn 

l^""; Jo Jo 

Then 

/■°° /■°° 1 Id d 
/(0,...,0)=/ .../ F(pi,...,p„)dpi...dp„ 

Jo Jo Pi Pn Opi dpi 

The problem is translation invariant, therefore we can get the value of / at any 
point. 



Inversion 3. The usual inversion formula also can be applied to the Lagrangian 
Radon transform. Denote by G{v) the average of = Ti-compf over all affine 
Lagrangian subspaces containing a point v. Denote by (t„ the area of (fc — 1)- 
dimensional unit sphere, 

'^'^ ~ r{n/2) 

Lemma 7.1. 

G{v)^— I f{w)\v-w\~'^dv (45) 

f2n JR2.1 

Proof. Denote by dv^ the measure dpL^ regarded as a measure on the whole 
R^". We consider the average of all dv^ for L passing through v and integrate / 
with respect to this measure □ 

The integral equation ps)) is well known, it can be solved in terms of the Riesz 
potentials, see, e.g. [8j. Consider the operator 

2~"7rr(n - q;/2) f |_2„+a s , 

r(a/2) Jk2" 

For a fixed / G iS(R^"), we get a function I^f admitting a meromorphic continua- 
tion in a to C with (possible) poles at a e 2n + 2Z+. Then 

I-nInf = f, fe 5(R2"), 

and this is the formula reconstructing / from G in (|45p . Also, positive integer 
powers of the Laplace operator A can be expressed as 

In particular, for even n, 

/ = const • A"/^^ 



THE LAGRANGIAN RADON TRANSFORM AND THE WEIL REPRESENTATION 23 



8. Radon transform of distributions 
Counterexample. Consider the function 7(0;) on M^" given by 

^^^^ " (l + |x|)"ln(2 + |a;|) 

where \x\ denotes the distance between and x. Let h{r) be an arbitrary positive 
function on M+ such that 




r < 1; 
r > 2. 



The sequence 

7Ar(a;) := h{\x\/N)j{x) 
converges to 7(0;) in the space S'{M?"'). On the other hand the sequence 

T^complNiL) 

increases nionotonically and tends to 00 uniformly on compact sets. Therefore 
T^complN{L) has no Umit in S'{AC{n, M) (and also has no limit in the space £'{A{L) 
of all distributions). 

Notice that 7 € L^(R^"). Therefore we can not extend the Radon transform 
5(K2") -s- S{AC{n,M)) to a continuous operator L'^{m?'') £'{AC{n,^)). 

Radon transform of distributions. Denote by ^(IR^") the set of all C°° func- 
tions on M^" such that their partial derivatives satisfy the estimates 

H^{h) :=sup(l + |x|)"^/(x) <oo 

for any multiindex a = {a\, . . . , a^n)- We regard Ha{f) as seminorms in B{S?^). 
By S'(R2") c iS'(M2") we denote the space of all distributions, which are continuous 
functional on the space ^(M^"). 

By ACx denote the set of all L G AC(n, M) containing x. Consider the stabilizer 
Gx C U(n) K M^" of the point x. Denote by da^iL) the unique probability measure 
on ACx invariant with respect to Gx- 

Theorem 8.1. The Radon transform TZcomp '■ ^(R^") — > S{A£{n,R)) admits a 
unique continuous extension to an operator ^^'(K^") — >■ S' {AjC.{n,M.)) . For <p € 
S{A£{n,R)) we define the function a(p{x) on M^" given by 

aifiix) = / tp{L)dax{L), 

For f e and ip e S{A£in,R)) we set 

{Tlcompf,'^) = (/,a</?). 

Lemma 8.2. There is a unique up to a scalar factor U(n) k -invariant measure 
da on the set M. of pairs x G M^", L G AC{n,R) such that x G L. It is can be 
represented as 

da{x, L) = dax{L) dx = dii^ix) dL, 
where dL is the U(n) k R'^'^ -invariant measure on AjC{n,R). 
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Proof. The space 7W is U(n) k R^"-homogcncous, a stabilizer of a point is 
isomorphic to 0(n). Since the Haar measure on U(n) k M^" is two-side invariant and 
0(n) is compact, the quotient space (U(n) ix M^"')/0(n) admits a unique invariant 
measure. 

It is easy to verify that the measures dax{L) dx and dfiL{x) dL also are invariant. 

□ 

Corollary 8.3. For f e ^(M^"), ^ g AjC{n,R), we have 

/ T^compf{L)ip{L)dL = / f{x) / ip{L) dax{L) dx. 

Proposition 8.4. If ip G S{AC{2n)), then 

aip{x) = / (p{L)dax{L) = 0(|a;|~"), a; oo. 
JaCx 

Estimate of a^p. Denote by B[a;,r] C R^" the ball with center x of radius r. 

Denote by Tr{x) the set of all subspaces L e AjCx, such that B[0, r] (1 L ^ 0. 
Then ax{Sr{x)) depends only on p = r/\x\. Denote 

h{p) := axiVrix)) 

Lemma 8.5. The function h{p) monotonically increases, h{0) = 0, h{l) = 1, and 

h{e) = Ce" + o(e"), e 0. 

Proof. Only the last statement requires a proof. Consider the product S^"~^ x 
£(n,M), where S^"~-^ is the unit sphere. Equip this space with a natural proba- 
bilistic measure. Fix xq e S^"~^ and Lq S £(n, M). Let 

— S c S2"-i X £{n,R) be the set of all pairs x e S^""!, L e £{n) such that 

L n B[x,r] 7^ 0; 

— Ac C{n, R) be the set of all L e L{n, M) such that L n B[xo, r] ^ 0; 

— S c S^"-! be the set of all x e S^""! such that Lq n B[a;, r] ^ 0. 
Then 

{measure of A} = {measure of S} = {measure of S} 

Let us equip the unit sphere S^"~^ with the intrinsic metric. The set S C S^"~^ is 
neighborhood of an (n — l)-dimensional equator of radius arcsinr. Its volume can 
be evaluate explicitly, but asymptotics of volume as r — >■ is obvious. □ 

Denote by 0^ the function on AC{n, M) given by 




1, if LnB[0,r] 7^ 0; 
0, if LnB[0,r] = 0. 



Lemma 8.6. For if e S{AC{n,R), 

aip{x) = 0{\x\~'^), x^oQ. 
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Proof. Denote by d{0, L) the distance between and a subspace L E AC{n, M). 
It can be easily shown that d{0, L)^ is a smooth function on AC{n, M). It is sufBcient 
to prove our statement for functions (p{L) having the form 

ip{L)^q{d{0,L)), 

where q{t) G 5(K) is an even and monotonically decreasing on [0,cxi). 

We represent f{L) as 

poo 

^{L)= / er{L){-q'{r))dr 
Jo 

and get 

/•oo p\x\ poo 

aLp{x)= I aQr{x){-q'{r))dr ^ / aer{x){-q' (r)) dr + / aQr{x){-q' (r)) dr 



|2 

We have Qrix) = 1 if < r. Therefore the second summand is 

{-q'{r))dr^qi\x\) 



l\x\ 

Next, we take 6 and C such that h{p) ^ Cp" ioT < p < 5 and spht [0, |a:;|] into 
two segments [0,5|a;|] and [S\x\, \x\] 

aer{x)(-q'{r))dr ^ / h{d/\x\){~q'{r))dr 
Jo 

pS\x\ />oo 

^ C|a;r" / r'\-q'{r)) dr ^ Clxp" / ^"(-(/'(r)) dr - jxp" 
Jo Jo 

Next, 

aQr{x){~q'{T))dT ^ / {-q'{r))dr ^q{\x\) + q{5\x\) 
I5\x\ JS\x\ 

Thus the asymptotics of our integral is ^ |a:|~". □ 

We also need in estimates of partial derivatives of functions a(p{x). Let ej be a 
basis vector in M^" 



d _ _ 
dxj '^'^^ ds 



[ Lp{L)da^+se.{L) = y- / ip{-se.j + L) d(Jx{L) 

•lAC^ + sa, s=0 -lAC^ 



s=0 



We have 

^ip{-se. +L)e S{AC{n,R)) e 5(yl/:(n,R)) 
ds s=o 

by definition of S{AC{n,M.)), and we get the same asymptotic. This finishes proof 

of Theorem O 



9. The image of Tlfiat 
Fourier transform. Consider the space SymmQ(n + 1) of real symmetric (n 



1) X (n + 1) matrices of the form ( n ) • -'^^ ^^^^ convenient for us to present 
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coordinates (T, r) on AC{n,R) in the matrix form 
The group GL(n, M) acts on Symmo(n + 1) by transformations 

We write the Fourier transform on Symmo(n + 1) by 

ip{E) = j FiM)e'^''^^,dM 

J SymniQ (n+1) 

Lemma 9.1. Let F be a tempered distribution on Symmg(n + 1) satisfying the 
determinantal system of equations. Then its Fourier transform is supported by 
matrices of rank < 2. 

Proof. Transforming the determinantal system to the language of the Fourier 
transform, we get 

P • <p = 0, (46) 
where P ranges in 3 x 3 minors of the matrix 

/ Xii ... Xin 



V6 ••• oy 



(47) 



Matrices of rank 2. Consider the set A C SymmQ(n + 1) consisting of matrices 

of rank ^ 2. 

Proposition 9.2. The set A is a union of two closed sets: 

— the (2n — 1) -dimensional set M, consisting of matrices of the form 

Oj' rk-^A 



— the 2n- dimensional set N consisting of matrices of the form 

/ yh + z*y 2^ 

where z, y - are vector- columns. 



(48) 



'X 0^ 

The intersection M r\ N consists of matrices ( ^ ^ ) such that X is sign- 



indefinite ofrkX ^ 1. 



Remark. An elementary model of such situation is the surface z{x -\-y ) = xy in 



M"^. It consists of the 2-dimensional ruled surface z = ^i^yi in the layer — 1 ^ z ^ 1 
and the line x = y = 0. Only the segment — 1 ^ z ^ 1 of the line a; = y = is 
contained in the closure of the surface (this is a line of self-intersection). □ 
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Proof. We reduce X to the diagonal form. The only possible variants of 
diagonal values are (±1, ±1, 0, . . . ), (±1,0,0...), (0,0,0,...). Next, 

















• 





l 


= det ( 


±1 






6 




I?. 


6 





f2 I c2 



Let X be in canonical form with diagonal values (1, 1, 0, 0). Then we easily see that 

e = o. 

Next, assume that £, 0- By an appropriate element of GL(n,R) we reduce it 
to the form I I . The only way to extend ^ to a matrix of rank ^ 2 is 



62 

63 



V 1 
We get an expression of form (|48p with 



1\ 




0/ 







\bnj 



□ 



We need in more details concerning the piece C A 
Lemma 9.3. a) Each matrix S € N can be represented as 



b) There is a unique Sh{n,R) -invariant measure on N, it is given by 
dyi . . . dyn dzi . . . dzn = 2"c?ui . . . dun dui . . . dun 

Proof, a) We set u = {z + y)/2, v = {z - y)/2. 

d) The group SL(n, M) acts on the manifold N as, y ^ gy, z 1— >■ gz. 



□ 



Description of the image of the Radon transform. Consider a function 
X{u,v) e 5(M" X M"). For a function ip £ 5(SymmQ(n + 1)) we consider the 
value 



I 







x{u, v) du dv 



(49) 



In fact, this is a tempered distribution supported by N . 
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Theorem 9.4. The following conditions are equivalent: 



F is contained in the image o/iS(]R^") under the Radon transform; 



F is a Fourier of a distribution ()49|) . 



Proof. Consider the Fourier transform of our distribution. 




dz. 



This expression is the Radon transform of the function 



= / x{x:z)e'^'' dx 



(50) 



Note that the the operator JSOl) is a bijection 5(R") 5(R"). 

Extraneous solution. Let ip he a. tempered distribution on SymmQ(ri + 1) sup- 
ported by A and satisfying the equations (|i5)) - ((Tf|) . Notice that any measure 
supported by A satisfies these equations. On the other hand, the algebraic variety 
A is singular, therefore some partial derivatives of measures also are admissible. 

Then the Fourier transform of is a solution of the determinantal system on 
Symmo(n+ 1). However, M and N produce two different families of such solutions. 
We also observe that the image of Fourier transform is not dense in topology of 
5'(SymmQ(n + 1) in the space of solutions of the determinantal system. 



Weil representation. Recall that the Weil representation We of Sp(2n,R) is 
given as follows (see, e.g., [22], Section 2.1. For generators of ASp(2rt,R) we have 



10. Decomposition in tensor product 




(53) 



(55) 



(51) 



(52) 



(54) 
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The group of operators generated by ([5T|) -([55 |) is isomorphic to a central extension 
of ASp(2n,R) by T = K/27rZ. The central subgroup T acts by multiplications by 

e^^ 9 eR. 

In fact we get a projective representation of ASp(2n,R), 

We{gi)We{g2) = (^{91,92) ■ 1^6(51,32), (56) 

where (^{91,92) is a scalar factor, \a\ — 1. There two reasons for appearance of this 
factor. 

First, we have the sign ± in (|5ip . In fact, operators ([5T|) - ([5^ determine a 
representation of two-sheet covering of Sp(2n,K) 

Secondly, the operators ([53| and ([54)) do not commute, they generate the Heisen- 
berg group. 

Tensor product. 

Theorem 10.1. The representation 0/ ASp(2n,R) on ^(R^") and on L'^iR'^'') 
is isomorphic to the tensor product of the Weil representation We with its com- 
plex conjugate representation We. The intertwining unitary operator L^(M^") — >■ 
L2(R") (g, is given by 



Hf{u,v) = 




This operator also is a bijection ^(M^") _> 5(M")(g)5(M"). 

Remarks, a) We recall that the suitably completed tensor product of spaces 
L^(M) (g) L'^{M) is canonically isomorphic to L'^{M x N). 

b) The operator _ff is a composition of the Fourier transform with respect to half 
of variables, 

J'fix, y) = J^^ fix, y) e'<^'«> dx 

and of the rotation of the plane [y, ^) by 45 degrees. 

c) We have 

{We{gi)®We{gi))-{We{g2)^We{92)) = ^(51, 52)^(91 , 92)(We{9i92)®We{gi92)) . 
Since \<t\ — 1, the scalar factor disappears and we get a linear representation. □ 

Proof is straightforward, we look to images of the operators (|5T |) -([55 |) under 
the partial Fourier transform and the rotation. □ 

11. Pair of wedges 

Siegel half-plane. Denote by Z+(n) the space oinxn matrices P such that Im P 
is positive-definite (this set is called Siegel upper half-plane). By {n) we denote 
the space of matrices with negative-definite ImP. The group Sp(2ri,R) acts on 
Z'^{n) by linear fractional transformations 

P ^ {aP + b){cP + dy'^ (57) 
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Note that a complex matrix P determines a point of >C(n,C) and the formula (|57p 
is a special case of ((22|). Note that this expression is defined everywhere, 

More generally, consider the set AZ^{n) :— Z^{n) x C", denote its points by 
(P, tt). The group ASp(2n,R) acts on this space by transformations 

(P,7r) K> ((aP + &)(cP + d)-\ OTT + r - (aP + 6)(cF + d)^^(c7r + s)) (58) 

Geometric interpretation. As we have seen in Section 2 (see (|12l) . ((T3| ) the 

complex space C'^" is equipped with two forms, the skew-symmetric form A and 
the indefinite Hermitian form M. Consider the action of the real group Sp(2n,M) 
on the complex Lagrangian Grassmannian C). Evidently, inertia indices of M 
on L G £(n, C) are invariants of this action. Denote by £(n,C)+ (resp £(n,C)_) 
the set of Lagrangian subspaces strictly positive (resp. negative) with respect to 
M. Denote by AC{n,C)± the set of subspaces of form ^ + i, where £ C^", 
L e C{n,C)+. 

We represent such subspace as a graph of a map W- — >■ W+ as in ([T5|) and get 
the formula 



Another realization of the Weil representation. Let / be a function on M". 
We define an integral transform K by 

i?/(P, TT) ^^-L-^ £ /(u)e^"'^-+7f du. (59) 

Obviously, for any / G 5'(R") we get a holomorphic function on AZ^{n). 

This operator sends the space L^(M") to a certain Hilbert H space of holomorphic 
functions on AZ^{n). We wish to discuss this space and the action of the group U 
of the group in this space. 



Reproducing kernel. First, our Hilbert space can be easily described on the stan- 
dard language of reproducing kernels. Consider a Hilbert space H whose elements 
are continuous functions on a domain 57 C M.^ . Assume that a linear functionals 
F H> /(a) are continuous on H for all F E fl. The for each a E fl we have 

./(a) = {.f,'Pa)H 

for some ipa G H. The function 



K{a,b) = {ipa,^b) = ^aib) = (Pbia) 

is called the reproducing kernel of the space H. The space H (the set of function 
and the inner product) is uniquely determined by the reproducing kernel, see, e.g. 
[22], Section 7.1. 

Proposition 11.1. In our case, 
K{P,tt; R,p) = 

= (^fl,p(P, ^) det[(-^(F - i?))-i/2] exp {-z(^ - p)*(F - R)~\n - p)} . (60) 
We choose a holomorphic branch of the determinant by 

det[(-z(F-i?))-i/2] _ =Aet[{~i-i)-^/^]^l 

P-Q—i 
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Proof. We set 



I j 

^rA^) = exp{ -x*Px + W'x} 



27r"/2 "-^^2" 
It is contained in L2(R") if ImP > 0. Then 

Kf{R,p) = (/,$fl,p>L2(R„) = {Kf.ipR^H 
and come to the reproducing property. 



□ 



Differential equations. 

Theorem 11.2. Functions .^/(P, tt) e H{AZ^{n) satisfy the equations 

DSiF{P, tt) = 0, 
where D ranges in all the 2x2 minors of the matrix 

dp2i 



d 

d 



a 


a 






dp22 


ap2n 


a 


2 ^ 


apg.2 




aTT2 


a'Kn 



a-rri » 

_a_ 

a-n2 



a 

a^^n 



Proof, a) Indeed the functions ipx{P,Tr) = satisfy these equa- 



tions. Next, we differentiate the integral ((59|) with respect to the parameters P, tt 
and get the required statement. However some justification here is necessary. We 
refer to Theorem 7.7.6 [211 . □ 



Note that the image of any tempered distribution under the transform A satisfies 
the same system of differential equations. 



The action of the group ASp(2n, M) in the space of holomorphic functions. 

First, we transfer the operators of Sp(2n,R) 



Lemma 11.3. The group ASp(2n,M) acts in the space H by the formulas 



5 I 



ON 



P(P,7r) = ±det(a)-i/2(a)p(a*-ipa-i,a*-i7r) 



(61) 
(62) 



1^ 

/I b 0\ 

SlO 1 P(P,7r)=P(P + 6,7r) 

\0 ij 

1 o\ 

S \-l P(P,7r) = det{-P)-^/^e-i-^'^''''F{-P-\p-^TT) (63) 

1/ 



'1 0> 
S \ 1 I P(P, tt) 
.0 s 1. 



(64) 
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^1 0> 

S\0 1 I F(P,7r) = F(P,7r + r) (65) 
,r 1, 

This is proved by straightforward calculations. 

Product of wedges. We can apply the same construction to the complex con- 
jugate of the Weil representation the Weil representation. We write the integral 
operator 

to the space T-L- of holomorphic functions on AZ~(n). All formulas are the same, 
only i in all expressions must be replaced by — i. 

Our representation of ASp(2n, M) is a tensor product of the Weil representation 
and the complex conjugate representation. Therefore we can realize in the space 
V.^ (8 'H~ holomorphic functions on AZ^{n) x AZ~{n) 

Intertwining operator. 

Theorem 11.4. The operator intertwining the representation o/ ASp(2n, M) in 
and in H'^ is given by the formula 

y) 



y{P, tt; Q, x) = -i- det[(-i(Q - P))-^'^] [ f{x, 

f » ,t f 2(P-Q)-'' (P-Q)-'(P + Q 



dx dy 



Wc omit a formal proof. It is sufficient to verify that the operator commutes 
with generators of ASp(2n, R). 

Additional remarks. We note that the matrix 

/ 2(P-0)-i {P-Q)-\P + Q) 

/ 1' 

is complex symplcctic. Moreover, the eigenvalues of \ g are ±1, and 



eigenspaces are elements of ^£(n,C)+ and AL{n,C)- with coordinates P, Q. 

The Radon transform corresponds to the case Q = P (this corresponds to the 
common edge of two wedges) . 
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